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ABSTRACT 



In 1989 1 Saddleback College's (SC's 



) English 



composition assessment process was evaluated to determine its success 
in placing students in appropriate courses. To gain entrance to one 
Of SC's three composition courses, students -must complete a 
prerequisite cou'^se or take the College English Placement Test (CEPT) 
and, at the option of the student, provide a writing sample. The 
three courses available are a university-level composition course, a 
composition class that satisfies the composition requirement for 
graduation with an associate degree, and a remedial/basic skills 
course. Random samples were chosen from the 2,714 students who had 
taken the CEPT between July 5, 1988 and January 31, 1989 from the 
students who provided a writing sample, and from the students who 
were not placed with the CEPT. Study findings, based or a comparison 
of placement results and course success, include the following: (1) 
students who enrolled in a course more advanced than that suggested 
by their CEPT score succeeded at a higher rate than the group that 
took the recommended course; this may be due to the fact that a 
student would normally have needea to take the writing sample to 
qualify for placement into the more advanced class; (2) of the 244 
students who took the writing sample, 94% were successful in English 
composition; the success rate for all students in the composition 
courses was 69%; (3) the success rates for CEPT-based placement 
students and students who took the prerequisite courses were about 
the same; (4) the success of the students placed through the 
assessment process indicated that the cutoff scores for the CEPT were 
not too low, and that they may have been too high; and (5) the 
English composition faculty did not recommend a change in the minimum 
testing levels due to their experience with the results of student 
placement based on the CEPT scores. An appendix provides CEPT scores 
and data on course success. (JMC) 
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A Review of the English Composition Assessment of 
Saddleback College Students through 
the Matriculation Program 

by 

Steven Sworder, Ph.D., Ed.D. 
Matriculation Research Specialist 



Summary i->f QhservatlQn..^ and RennmmftndatiQn i 

1. Students who complete the writing sample are a very motivated 
group. They successfully complete the course in which they 
enroll ninety-four percent of the time. This compares with 
an average completion rate for all English composition 
students of sixty-nine percent. 

2. The assessment process provides students with course 
placements that match the placements of continuing students 
resulting from completion of prerequisite courses. The 
likelihood of success of a student who entered a particular 
composition course, after she or he received that placement 
level from the assessment process, is the same as that for a 
student who enrolled after having completed the prerequisite 
course . 

3. Composition course placement based on the writing sample is 
independent of the placement that results from the objective 
test. In other words, .a student originally placed in the 
remedial level composition course because of his or her 
performance on the objective test, had the same chance of 
-^eing placed in the transfer level ccurse after completing 
the writing sample as a student originally placed in the 
associate degree level course. Because of this independence, 
any student who is not satisfied with his or her course 
placement following the objective test, no matter the level 
of that placement, should be encouraged to complete the 
writing sample. 

4. It is clear that the objective test cut off scores are not 
too low. Students who enroll in the recommended courses have 
a high success rate. While it is not possible to use the 
results of this study to determine that the objective test 
cut off scores are not too high, there is reason to suppose 
that they are not. The determination of these scores 
followed a period of careful consideration by the English 
composition faculty. This was the same process used by the 
mathematics faculty to set the cut off scores for the 
assessment tests in that discipline and rhose scores were 
shown to be appropriate in a similar study to the one 
reported here. Because on their experience with the results 
of student placement based on these scores, the English 
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composition faculty feel the current minimum levels are 
appropriate. Further, the students do not appear to be 
dissatisfied. For those students who feel that their 
composition skills have not been accurately assessed, a well 
defined appeal process is in place. Consequently, no 
recommendations for change in the cut off scores of the 
objective portion of the English composition assessment 
process resulted from this investigation. 
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A Review of the English Composition Assessment of 
Saddleback College Students through 
the Matriculation Program 

by 

S.C. Sworder, Ph.D., Ed.D. 
Matriculation Research Specialist 

Introduction 

The English composition curriculum at Saddleback College is 
divided into three levels. The transfer level course (English lA 
— Principles oil Composition I) is generally accepted as 
equivalent to the first college level composition course by the 
universities in California. One level below the transfer course 
is a composition class (English 200 — Fundamentals of 
Composition) that satisfies the composit n requirement for 
graduation with an Associate in Arts (A.i. degree. Two levels 
belov/ the transfer course is the remedial/basic skills course 
(English 300 — Beginning Writing) . 

To gain entrance to a particular English composition course 
at Saddleback College students must generally either complete the 
prerequisite course or receive the desired placement through the 
English composition assessment process. This process consists of 
an objective test, the College English Placement Test (hereafter 
referred to as the CEPT) , and, at the option of the student, a 
writinc sample. If the student is satisfied with the placement 
indicated by the CEPT score, the student is given permission to 
enroll in that course. The minimum scores are seventy for the 
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transfer course and fifty-five for the A. A. level class. If the 
student is dissatisfied with the initial placement, then she/he 
may complete a writing sample. If the placement resulting from 
this exercise is higher, then she/he is given permission to enroll 
in the more advanced course. If the placement is lower, she/he 
may still exercise the placement recommendation from the CEPT. If 
still not satisfied with the placement level, the student may 
enter into an appeal process that will result in a review of their 
situation and an individualized composition course placement. 

Assessment Hslng tht^ College English 
Planement Test (CEPT) 

A frequency distribution for scores achieved by the 2714 
individuals who completed the CEPT between July 5, 1988 and 
January 31, 1989 was determined and placed in figure 1. This 
frequency distribution is, as expected, approximately Normal (i.e. 
bell shaped) . The uncharacteristically large frequency spike 
corresponding to a CEPl score of seventy was due to the fact that 
this was the minimum acceptable score for placement into the 
transfer level course. Students who scored below seventy, but had 
reading scores that suggested to the English assessment 
coordinator that the student was likely to be successful in this 
level of composition, were reassigned a score of seventy for the 
CEPT. A similar explanation existed for the peak at fifty-five, 
since this was the minimum score for admission to the A. A. level 
composition course. 
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Of the 2714 individuals who took the CEPT during the period 
of study, forty-six percent were placed at the remedial level ► 
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Figure 1 
Frequency Distribution 
of CEPT Scores 
(N=2714) 

The cut off for the transfer level course was at the eighty-second 
percentile and thus thirty-six percent of the individuals were 
placed into the A. A. course. The distribution of course 
placements based on the CEPT score was placed in figure 2. 

A comparison of CEPT ha:;ed placement and course success was 
made. A search was conducted through the transcripts of thirty- 
six percent of the 2714 students who had taken the CEPT between 
July 5, 1988 and January 31, 1989. These students were selectee' 
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Figure 2 

Distribution of Composition Course 
Placements Based on the 
CEPT Score Received 
(N = 2714) 

arbitrarily in an attempt to gather results from students with a 
wide range of CEPT scores. Students who received a grade of A, B, 
C, or Cr (Credit) were classified as successful and all others 
were classified as unsuccessful (i.e. those students who withdrew 
or received a grade of D, F, or NCr (No Credit) ) . It was found 
that fifty-two percent (505 individuals) had taken one of the 
English composition courses in either the Fall, 1988 term or 
Spring, 1989 term. The success rate (i.e. percent of those who 
enrolled who were ultimately successful) of those students who 
took the course into which they were placed based on the CEPT 
score was calculated as well as the success rates for those who 
enrolled in other levels. These results were placed in Table 1. 
The data on which those calculations were based was attached as an 
Appendix . 
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Table 1 



Comparison of CEPT Based Placement and 
Course Success 
(N=505) 



Course Actually Taken 



CEPT Course 
Placement 



Remedial 
% Successful 



A. A. 

% Successful 



Transfer 
% Successful 



Remedial 



76% (N=136) 



83% (N=23) 



95% (N=20) 



A. A. 



63% (N=8) 



78% (N=165) 



94% (N=31) 



Transfer 



100% (N=l) 



100% (N=l) 



75% (N-120) 



Those students who enrolled in a course more advanced than 
that suggested by their CEPT score succeeded at a higher rate than 
the group that took the recommended course. This may have been 
due to the fact that a student would normally have needed to take 
a writing sample to qualify for placement into the more advanced 
class. Since ninety-four percent of the students who took the 
writing sample and then enrolled in English composition were 
successful (see Table 4), it was not surprising that these 
students exhibited the high success rate displayed Table 1. These 
students may, perhaps, be characterized as uncommonly motivated. 



A total of 244 individuals took the writing sample during the 
investigation period. The distribution of the course placements 
that resulted from the performance of these students on the 
writing sample was shown in figure 3. While slightly more than 
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half of those who took the writing sample received placements into 
the A. A. level course, more than one-third were placed into the 
transfer course. 




Figure 3 

Distribution of Course Placements 
Based on the Writing Sample 
(N=244> 

The rate of placement in the transfer course from the writing 
sample was twice that from the CEPT test. The placement rate in 
the remedial level course, based on the writing sample, was 
approximately one-quarter that observed for the CEPT. A direct 
comparison of the course placement rates for these two tests was 
constructed in graphical form and placed in figure A. 

The significant difference in placement rates may well have 
been due to the difference in the populations. All students were 
expected to take the CEPT while only those not satisfied with 
their placement needed to take the writing sample. It was likely 
that the large number of students who placed into the remedial 



10 



7 



CEPT (N=2714) B Writing Sample 

(N«244) 




Remedial A. A. Transfer 

Composition Course Level 

Figure 4 

Rate of Course Placements as a Function of the 
Assessment Instrument 



course based on the CEPT were comfortable with that result. Th(5 
students who took the writing sample obviously believed their CEPT 
placement was too low and may have demonstrated that fact on the 
writing sample. 

It was possible to determine the CEPT score for fifty-six 
percent of the students who completed the writing sample. For 
those 137 students, the distribution of course placements 
resulting 'rom the writing sample was essentially the same as that 
shown in figure 3 for the entire population of 244. Specifically, 
for these 137 students, fourteen percent were placed in the 
remedial course, forty-eight percent were placed in the A. A. 
course, and thirty-eight percent received the recommendation to 
enroll in the transfer level class. 

ERIC 11 
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A comparison was made betw«.>en the English composition course 
placement recommendations that resulted from the two different 
instruments for this group of 137 students. The two placements 
were in agreement for thirty-five percent of those choosing to 
take the writing sample. Those who received a higher placement 
due to the writing sample were sixty percent of those who chose to 
complete that exercise. Five percent received a lower placement. 
These results were summarized in Table 2 . 



Table 2 

Comparison of Placement Recommendations of Those 
Students Who Completed Both the CEPT and 
the Writing Sample 
(N=137) 



Writing Sample 
Placement 



CEPT Placement 



Remedial 



A. A. 



Transfer 



Remedial 
A. A. 

Transfer 



9% 

23% 
16% 



5% 

25% 

21% 



none 
none 
1% 



Ignoring the two students who completed the writing sample 
after they already had received English lA placement through the 
CEPT, the independence of the course placements based on these two 
instruments was tested using a standard contingency table 
technique . The observed frequencies in each category were shown 
in Table 3 along with the predicted frequencies, in parentheses, 
based on the assumption of independence. 
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Table 3 



Contingency Table for Comparison of Placement 
Recommendations of those Students Who 
Completed Both the CEPT and 
the Writing Sample 
(N-135) 



CEPT Placement 



Writing Sample 
Placement 



Remedial 



Remedial 



12 
(9.15) 



7 

(9.85) 



31 
(31.3) 



34 
(33.7) 



Transfer 



22 
(24.6) 



29 
(26.4) 



The differences between the observed and expected frequencies 
were determined to not be of statistical significance and thus 
most likely due to chance. There was sufficient evidence to 
support the claim that the placement recommendations provided by 
the writing sample and CEPT were independent. In other words, the 
likelihood of a student who received a CEPT placement into the 
remedial course being placed in the transfer level course after 
the writing sample was completed was the same as that for a 
student who had originally been placed in the A. A. level course. 
Similarly, the likelihood of a student who received a CEPT 
placement into the A. A. course being placed in the remedial level 
course after the writing sample was completed was the same as that 
for a student who had originally been placed in the remedial level 
course. Because of this independence any student not satisfied 
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with their course placement, no matter the level, should be 
encouraged to complete the writing sample. 

Compariso n of Writing Sample Placement 
and Coursp Success 

To determine the relationship between the writing sample 

placement into the English composition curriculum and course 

success, a search was conducted through the transcripts of all 

students who had taken the writing sample between July 5, 1988 and 

January 31, 1989 and for whom a CEPT score was known. Students 

who received a grade of A, B, C, or Cr were classified as 

successful and all others were classified as unsuccessful (i.e. 

those students who withdrew or received a grade of D, F, or NCr) . 

The success rates for the one hundred students who actually 

enrolled in a composition course was displayed in Table 4 . 



Table 4 



Comparison of Writing Sample Placement 
and Course Success 
(N=100) 



Course Actually Taken 



Writing Sample 
Placement 



Remedial 
% Successful 



1\, • • 

% Successful 



Transfer 
% Successful 



Remedial 



100% (N=7) 



100% (N=4) 



100% (N=3) 



A. A. 



100% (N=5) 



87% (N=38) 



100% (N=3) 



Transfer 



None 



100% (N=l) 



97% (N«39) 



14 
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Ninety-four percent of the students who took the writing 
sample and then enrolled in English composition were successful. 
This completion rate was significantly higher than the sixty-nine 
percent rate for all students in these composition courses. 
Students who requested and then completed the writing sample may, 
perhaps, be characterized as uncommonly motivated. 

rnmpariRnn of the Suncft ftfi of Stndftnta Who Were Admitted 
After Com plAi-lng the Prerequisite Course and 
Those Admltteri Through Ot-hP>r Means 

A standard statistical test for independence was conducted. to 
determine if student success was independent of the path used to 
enroll in an English composition course. The student may have 
entered the course after completing the prerequisite course at 
Saddleback College or through use of a test score, course at 
another institution, or by some other means. To provide the 
necessary data for this test a random sample of two hundred 
students was selected. Using a random number table and the 
complete set of grade sheets for the composition courses ior the 
Fall, 1988 and Spring, 1989 terms one hundred students at the 
A. A. level and one hundred students at the transfer level were 
selected. The transcripts for these students were examined to 
determine if they had completed a prerequisite English composition 
course at Saddleback College prior to enrolling in the course from 
which they were selected. Further, students who received a grade 
of A, B, C, or Cr in the class from which they were chosen were 
classified as successful and all others were classified as 
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unsuccessful (i.e. those students who withdrew or received a grade 
of D, F, or NCr) . 

Of the sample of two hundred students, forty-three percent 
had oompleted a prerequisite course at Saddleback College, Of 
this iiample of two hundred students, seventy-two percent were 
successful. Since the success rate for all students at the A. A. 
level was sixty-eight percent and the success rate for all 
students at the transfer level was seventy-two percent, the 
success rate observed in this sample was essentially the same as 
the population from which it was drawn. The sample was thus found 
to be representative of the population of composition students at 
the upper two levels. 

The students in the sample were divided into categories 
depending on their method ot entry into the class and their 
success in that class . The independence of course success and 
these two methods of course entry were tested using a standard 
contingency table technique. The observed frequencies for each 
category were shown in Table 5 along with the predicted 
frequencies, in parentheses, based on the assumption of 
independence. The differences between the observed and expected 
frequencies were determined to not be of statistical significance 
and thus most likely due to chance. 

There was sufficient evidence to support the claim that 
student success was independent of the route used by the student 
to gain entry into the composition class. The likelihood of 
success of a student who entered a particular composition course, 
after she or he received that placement level from the assessment 

ERIC 16 
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process, was the same as that for a student who enrolled after 
having completed the prerequisite course. The assessment process 
appropriately matched the preparation given to continuing students 
through prerequisite courses . 

Table 5 

Contingency Table 
Course Success vs. Method of Entry 



Method of 
Entry 



Course Success 
Number of Successful Number of Unsuccessful 
Students Students 



Prerequisite 
Course 



60 
(62) 



26 
(24) 



Placement Test or 
Other Means 



84 
(82) 



30 
(32) 



It was difficult, from a research standpoint, to address the 
validity of a required placement device already in place. The 
very presence of required placement effected the nature of the 
data available for collection. For example, if the goal of the 
placement process was to select students for entry into various 
classes based on certain percentile level.=?, theoretically figure 1 
could have been used to determine the appropriate cut off scores. 
But, because students were required to obtain certain scores for 
course placement at the time the CEPT was taken, modifications 
were made to the data to address the needs of those individuals 
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close to the existing minimum scores. S* adents were reassigned 
scores of seventy or fifty-five to allow entry into a more 
advanced class for which, it was felt, they would most probably be 
successful. However, information about the actual scores for 
these students was lost and the determination of the actual 
percentiles disturbed. 

If, on the other hand, the purpose of the assessment process 
was to place students into the most advanced course for which they 
would probably be successful, from a research standpoint (though, 
perhaps, not a humanistic one) it would be desirable to allow 
students, if they so desired, to enter classes for which the 
faculty felt they were not prepared and actually observe that they 
were unsuccessful. Although this study has documented the success 
of students in courses more advanced than those recommended by the 
CEPT or by the writing sample, the fact remains that to gain entry 
into these more advanced classes the students generally had to be 
placed there based on the recommendation of one of the two 
instruments. Insufficient data exists on the success or lack of 
success of students who entered a class for which neither 
instrument suggested success. 

It was clear from the success of students placed through the 
assessment process that the existent cut off scores were not too 
low. That they were not too high (in the context of the assumed 
goal of the second example) was not equally clear. To permit 
research in this area it would be necessary to allow the entry 
scores to decrease until some statistically significant disaster 
became apparent . This could be done by lowering the cut off 
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scores sliyhtly each semester while student performance v\?as 
monitored. This process would continue until a significant 
decrease in student success was observed. At that point, it would 
be known that the immediately preceding scores were the minimum 
consistent with the assessment policy of the matriculation 
program. 

Conclusion 

While it is not possible, under the current assessment model, 
to quantitatively show that the cut off scores are not too high, 
there is reason to believe that this is the case. These minimum 
performance levels on the CEPT were set after careful 
consideration by the English composition faculty. A similar 
process was used by the mathematics faculty to determine the cut 
off scores for the mathematics assessment tests. A self -placement 
model in that discipline allowed for a statistical study of the 
validity of those values and they were found to be reasonable. 
Because on their experience with the results of student placement 
based on these scores, the English composition faculty feel the 
current minimum levels are appropriate. Further, the students do 
not appear to be dissatisfied. For those students who feel that 
their composition skills have not been acc.rdtely assessed, a well 
defined appeal process is in place. Consequently, no 
recommendations for change in the cut off scores of the objective 
portion of the English composition assessment process resulted 
from this investigation. 

o 19 
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APPENDIX 



CEPT SCORES AND COURSE SUCCESS 



In the table below is shown the English composition course 
success for 505 arbitrarily selected students for whom the CEPT 
score was known. The major column headings indicate the course 
level in which the student enrolled: remedial, A. A., transfer. 
The column subheadings indicate whether or not the student was 
successful in that course. Students who received a grade of A, B, 
C, or Cr were classified as successful (column heading: S) and all 
others were classified as not successful (column heading: NS) . 
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0 














0 


1 














0 


2 














0 


3 














0 


4 














0 


5 


1 












1 


6 














0 


7 














0 


8 














0 


9 














0 


10 




2 










2 


1 1 














0 


12 














0 


13 












1 


1 


14 














0 


15 








4 
1 






1 


16 














0 


17 








1 






1 


18 












1 


1 


19 








1 






1 


20 














0 


21 














0 


22 














0 


23 




2 










2 


24 




2 










3 


25 














0 


26 




2 










3 


27 




1 






1 




2 


28 




4 








1 


6 


29 




2 










4 


30 








1 






2 


31 




1 










1 


32 


3 


4 










7 


33 




4 










5 


34 




1 










2 



CEPT 


Remedial 1 




SOOFE 


NS 


S 


NS 


S 


NS 


S 


Total 


35 




3 










3 


36 




1 








1 


2 


37 


2 


3 










5 


38 


1 


2 










3 


39 


1 


2 










3 


40 




2 








1 


3 


41 




2 








1 


3 


42 




3 




1 






4 


43 




3 




1 






4 


44 


2 


6 








1 


9 


45 


1 


1 




1 




1 


4 


46 




2 










2 


47 




1 




1 






2 


48 




4 








1 


5 


49 




4 








1 


5 


50 


2 


7 


1 


2 




1 


13 


51 


2 


5 


3 


2 




2 


14 


52 


3 


1 2 




2 




2 


1 9 


53 


3 


9 




2 




3 


17 


54 


4 


6 




3 




1 


14 


55 




1 


3 


23 


2 




29 


56 




2 


6 


15 




2 


25 


57 


3 


1 


8 


11 




1 


24 


58 




1 


3 


13 




1 


1 8 


59 






1 


4 




3 


8 


60 






2 


6 




1 


9 


61 








6 




2 


8 


62 








6 




1 


7 


63 






2 


2 




3 


7 


64 








2 




3 


5 


65 






3 


5 




3 


1 1 


66 






3 


9 




1 


13 


67 






2 


1 1 




2 


15 


68 






2 


9 




2 


13 


69 






2 


6 




4 


1 2 



ERIC 



20 



17 



70 




1 




1 


8 


21 


31 


71 










6 


19 


25 


72 










2 


10 


12 


73 










4 


5 


9 


74 










1 


3 


4 


75 












7 


7 


76 












5 


5 


77 














0 


78 










2 


2 


4 


79 










1 




1 


80 










1 


3 


4 


81 










1 




1 


82 














0 


83 












1 


1 


84 












2 


2 


85 










1 




1 


86 












3 


3 


87 














0 


88 












1 


1 


89 










1 


2 


3 


90 










2 




2 


91 












1 


1 


92 














0 


93 














0 


94 












1 


1 


95 












1 


1 


96 












1 




97 












1 


1 


98 












1 


1 


99 














0 


100 














0 


Total 


36 


109 


41 


148 


33 


138 


505 



ERIC 



ERIC Clearinghouse for 
Junior Colleges 



